Method for optimizing image motion estimation

ABSTRACT

A method is provided for performing motion estimation in a system having a test image and a plurality of candidate images. A candidate image is selected and the difference between the test image and the selected candidate image is determined. The motion of an image is estimated according to this differencing and a determination is made of the duration of the motion estimation process in the system of the present invention. The candidate image selection, the differencing and the motion estimation are then repeated according to the duration determination. The duration determination may be a determination of a time duration or a determination of a number of machine cycles. The system is adapted to iteratively decrease a measurement of the error between the test image and selected candidate images as these actions are repeated. When the error stops decreasing and begins increasing the assumption is made in the system of the present invention that a best match has been determined. Thus a best match is iteratively determined unless a time out occurs first.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the field of video processing and, inparticular, to the compression and decompression of video signals.

2. Background Art Statement

It is well known to perform loop filtering within video compression anddecompression these systems. For example, it is known to provide atwo-dimensional spatial filter which operates on pels within a predictedeight-by-eight block. The filter is separable into two one-dimensionalfunctions, horizontal and vertical. Both the horizontal function and thevertical function are nonrecursive with coefficients of one-quarter,one-half and one-quarter except at block edges where one of the tagswould fall outside the block. In such cases the one-dimensional filteris altered to have coefficients zero, one and zero. Full arithmeticprecision is retained with rounding to eight bit integer values at thetwo-dimensional filter output.

In addition, it is well known to provide quantization within thesesystems. In a typical system the number of quantizations may be one forthe intrablock encoded DC coefficient and thirty-one for all othercoefficients. Within a macroblock the same quantization is used for allcoefficients except the intrablock encoded DC quantization. The decisionlevels may not be defined. The intrablock encoded dc coefficient isnominally the transform value linearly quantized with a step size ofeight and no dead zone. Each of the other thirty-one quantizations isalso nominally linear but with a central dead zone around zero and witha step size of an even value in the range two to sixty-two. In thesesystems the full dynamic range of the transformed coefficients cannot berepresented for smaller quantization step sizes.

To prevent quantization distortion of transformed coefficient amplitudescausing arithmetic overflow in the encoder and decoder loops, clippingfunctions are sometimes inserted. The clipping functions are applied tothe reconstructed image which is formed by summing the prediction andthe prediction error as modified by the coding process. This clipperoperates on resulting pel values less than zero or greater than twohundred fifty-five, changing them to zero and two hundred fifty-fiverespectively.

Values that are quantized in this manner may be dequantized in thefollowing manner. For all coefficients other than the intrablock encodedDC quantization the reconstruction levels, REC, are in the range of-2048 to 2047 and are given by clipping the results of the followingequations: ##EQU1## Where QUANT ranges from one to thirty-one. Thesereconstruction levels are symmetrical with respect to the sign of LEVELexcept for the values 2047 and -2048.

In the case of blocks which are intrablock encoded the first coefficientis nominally the transform DC value linearly quantized with a step sizeof eight and no dead zone. The resulting values are represented witheight bits. A nominally black block provides the value 0001 0000 and anominally white block yields 1110 1011. The codes 0000 0000 and 10000000 are not used. The reconstruction level of 1024 is coded as 11111111. Coefficients after the last non-zero one are not transmitted.

It is also know to provide both hardware and software forward andinverse discrete cosine transforms in these systems. When hardware isprovided for this purpose space is wasted on the integrated circuit chipbecause only one transform is performed at a time. Thus space on thechip is always taken up by a transform circuit which is not in use.

It is common to perform these discrete cosine transforms using a numberof multipliers and adders. For example it is known to perform an eightpoint fast discrete cosine transform in a single clock cycle usingtwelve multipliers and twenty-nine adders. It is also known to performit in more clock cycles using less hardware. These different transformdevices are useful for different applications. For example many highquality video applications require great speed and a great deal of spacefor transform hardware may be provided. In other applications, forexample, video conferencing great speed is not required and it ispreferred to provide more efficient use of hardware application.

SUMMARY OF THE INVENTION

A method is provided for performing motion estimation in a system havinga test image and a plurality of candidate images. A candidate image isselected and the difference between the test image and the selectedcandidate image is determined. The motion of an image is estimatedaccording to this differencing and a determination is made of theduration of the motion estimation process in the system of the presentinvention. The candidate image selection, the differencing and themotion estimation are then repeated according to the durationdetermination. The duration determination may be a determination of atime duration or a determination of a number of machine cycles. Thesystem is adapted to iteratively decrease a measurement of the errorbetween the test image and selected candidate images as these actionsare repeated. When the error stops decreasing and begins increasing theassumption is made in the system of the present invention that a bestmatch has been determined. Thus a best match is iteratively determinedunless a time out occurs first.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 shows a block diagram representation of a video processing systemincluding a system and method for accelerating the compression anddecompression of digital video signals of the present invention.

FIG. 2 shows a block diagram representation of the pipelinedarchitecture of the system and method for accelerating the compressionand decompression of video digital signals of the present invention.

FIGS. 3A, B show a block diagram representation of the data flow of thesystem and method of FIG. 2 when it is adapted to encode digital videosignals and physical buffer memories for use in this process.

FIG. 4 shows a block diagram representation of a data structure for theinput circular buffer of the encode dataflow of FIG. 3.

FIG. 5 shows a block diagram representation of a data structure for theoutput circular buffer of the encode dataflow of FIG. 3.

FIG. 6 shows a block diagram representation of the dataflow of thesystem and method of FIG. 2 when it is adapted to decode digital videosignals.

FIG. 7 shows a block diagram representation of a data structure for theinput circular buffer of the decode dataflow of FIG. 6.

FIG. 8 shows a block diagram representation of a process flow forsynchronizing the motion estimation and the encoding process in thesystem and method of FIG. 2 as well as buffer pointers related thereto.

FIG. 9 shows a block diagram representation of a process flow forsynchronizing the decoding process in the system and method of FIG. 2 aswell as buffer pointers related thereto.

FIG. 10 shows a block diagram representation of the forward discretecosine transform computation flow of the encoding portion of the systemof FIG. 3.

FIG. 11 shows a block diagram representation of the inverse discretecosine transform computation flow of the decoding portion of the systemof FIG. 3.

FIG. 12 shows a block diagram representation of a device for selectablyperforming either the forward discrete cosine transform of FIG. 10 onthe inverse discrete cosine transform of FIG. 11.

FIG. 13 shows an arrangement of positions which may be selected by themotion estimation method represented by the flowcharts of FIGS. 14A-C.

FIGS. 14A-C show flowchart representations of a motion estimation methodand an alternate embodiment thereof for use in the data flow of FIG. 3.

FIG. 15 shows a block diagram representation of a dequantization systemwhich may be used within the system of FIG. 2.

FIG. 16 shows a more detailed block diagram representation of thevarious memories and controls associated with the bus interface of thepipelined architecture of system of FIG. 2.

FIG. 17 shoes a more detailed representation of a frame add/subtractsuitable for use in the pipelined architecture of FIG. 2.

FIG. 18 shows a state diagram representation of the operations of theselectable loop filter of FIG. 2.

FIG. 19 shows a more detailed block diagram representation of the adderunit of the selectable loop filter of FIG. 2.

FIG. 20 shows a more detailed representation of the address unit of theloop filter of FIG. 2.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to FIG. 1, there is shown remote video processing system100 including compression/decompression accelerator 120. The busstructure of remote video processing system 100, including address bus116 and data bus 118, permits easy interconnection of the components ofa multimedia display system without using a host processor bus. Forexample, the local video data of video camera 127 may be received byvideo processing system 100 by way of line 129 and captured, digitized,subsampled and scaled by video capture 128. The signals provided byvideo capture 126 may then be processed by system 100 for transmissionto merge logic video by way of system output line 132 and capture 128.All of these operations are performed by remote video processing system100 without use of a host processor bus or an industry standard bus suchas bus 140 which may couple remote system 100 to a host computer by wayof bus 138. Because system 100 interfaces a source of video data such asvideo camera 127 to bus 140, system 100 may be understood to be a remotevideo interface system 100.

Within video processing system 100 digital video processor 112 performssoftware processing while accelerator 120 does hardwired functions.Nucleus gate array 121 performs the required memory interface functions.For example, VRAM emulation block 124 makes nucleus gate array 121function like DRAM with respect to capture 128. Video processing system100 is therefore limited to the memory configurations supported bynucleus gate array 121.

In a typical configuration of remote video processing system 100 up tosixteen megabytes of address space may be supported. The first fifteenmegabytes of this address space may be reserved for DRAM, for example,DRAM 114, which is interfaced with nucleus gate array 121 by DRAMinterface 122. The upper one megabyte of the memory space of videoprocessing system 100 is reserved for communication between variousdevices which may be coupled to address bus 116 and data bus 118.Compression/decompression accelerator 120 may occupy the first onehundred twenty-eight kilobytes of the communication area in the upperone megabyte of memory space.

Compression/decompression accelerator 120 is not required to decode allof the available one hundred twenty-eight kilobyte address space becausethe area is well in excess of the requirement for an internal registermap. Accelerator 120 may be an initiator or a target in a bustransaction within remote video interface system 100. When accelerator120 initiates actions it fetches thirty-two bit words from memory. Whenaccelerator 120 is a target it responds to all scalar accesses oraddresses in its range.

The bus structure of remote video interface system 100 may use a daisychain priority scheme for bus arbitration. In this schemecompression/decompression accelerator 120 may request access to buses116, 118 through nucleus gate array 121. In the preferred embodimentaccelerator 120 is the last connection in the daisy chain it has thelowest priority. Thus, bursts initiated by accelerator 120 may beaborted in the middle of a burst by another device. This type of abortdue to another device typically occurs when a display system controlledby nucleus gate array 121 requires display data. Under suchcircumstances compression/decompression accelerator 120 must relinquishbuses 116, 118 before the beginning of the following cycle. Accelerator120 later requests buses 116, 118 and, after gaining access by thenormal arbitration process, resumes the interrupted burst beginning atthe last address accessed.

Referring now to FIG. 2, there is shown a more detailed block diagramrepresentation of compression/decompression accelerator 120 of thepresent invention within remote video interface system 100 having astraight pipeline architecture rather than shared resources.Compression/decompression accelerator system 120 may be selectablyoperated in an encode mode and in a decode mode in accordance withinternally generated digital control signals. These two modes areeffective to perform and thereby accelerate many encode operations anddecode operations performed upon data transmitted by way of data bus 118within video processing system 100. This assists in reaching the videodata processing speeds necessary in order to perform real time video.

Data transmitted by way of data bus 118 of remote video interface system100 is received by accelerator bus interface 200 ofcompression/decompression accelerator 120. All RAM within acceleration120 must read and write by way of accelerator bus interface 200. Whenaccelerator 120 functions in the encoder mode a previous image block, orreference frame, as received by accelerator bus interface 200 may bestored in loop filter memory 206 or previous block memory 206. This datamay be applied to selectable loop filter 210 for filtering prior tobeing applied to frame difference block 220 depending upon whetherselectable loop filter 210 is enabled. In the preferred embodimentselectable loop filter 210 may be formed of a small RAM with separateread and write ports, a three stage adder and feedback.

It will be understood that during normal operation of accelerator 120data is constantly applied to loop filter memory 206 and constantly readfrom filter memory 206. Thus there is a constant flow of data fromaccelerating bus interface 200 to frame difference block 220. Theenabling of selectable loop filter 210 is controlled by loop filtercontrol line 214. Selectable loop filter 210 of accelerator 120, whichis enabled by asserting a control signal by way of loop filter controlline 214, may be a two-dimensional 1-2-1 filter applied to the interiorof an eight-by-eight pixel block. In the preferred embodiment ofcompression/decompression accelerator 120 the filter coefficients ofloop filter 210 may be as shown in Table I.

                  TABLE I                                                         ______________________________________                                               1         2           1                                                       2         4           2                                                       1         2           1                                                ______________________________________                                    

Selectable loop filter 210 may be applied to various areas of the storedimage in loop filter memory 206. These areas are as shown in Table II.The outer rows and columns of an eight-by-eight pixel block from theimage being filtered within loop filter 210 are filtered only in onedirection except at the corners. The pixel positions receiving thisfiltering in only one direction are indicated by ones in Table II. Thepixels at the corners of the pixel block filtered are not affected byselectable loop filter 210. This is indicated in Table II by zeros. Theremaining pixels in the interior of the block are filtered in bothdirections. The pixels which are filtered in two directions areindicated by the twos in Table II.

                  TABLE II                                                        ______________________________________                                        0       1       1       1     1     1    1    0                               1       2       2       2     2     2    2    1                               1       2       2       2     2     2    2    1                               1       2       2       2     2     2    2    1                               1       2       2       2     2     2    2    1                               1       2       2       2     2     2    2    1                               1       2       2       2     2     2    2    1                               0       1       1       1     1     1    1    0                               ______________________________________                                    

The reference frame data of loop filter memory 206 is applied to framedifference block 220 by way of reference frame line 209 either with orwithout filtering by selectable loop filter 210. A frame subtraction isperformed in difference block 220 when compression/decompressionaccelerator 120 performs motion estimation encoding. In the subtractionof frame difference block 220, the information of reference frame line209 is subtracted from the current frame information on current frameline 205. The difference between the reference frame data of line 209and the current frame data of line 205 is then provided for encoding atthe output of frame difference block 220. The information regarding thecurrent frame line is received from bus interface 200 by way of currentframe memory 204 and line 202. This frame subtraction of framedifference block saturates the resulting image values to a signed numberbetween negative and positive two hundred fifty-five.

The output of frame difference block 220 is multiplexed within transformmultiplexer 224 with the information applied directly by way of line205. Transform multiplexer 224 is controlled by multiplexer control line225. It will be understood that the state of control line 225 dependsupon whether the image block being processed is intrablock encoded ormotion estimation encoded. The output of transform multiplexer 224 isthen encoded by means of a forward discrete cosine transform operationperformed within selectable discrete cosine transform block 230 withincompression/decompression accelerator 120. Selectable discrete cosinetransform block 230, or reconfigurable discrete cosine transform block230, may perform either a forward or an inverse discrete cosinetransform. The transform performed depends on the control signalsapplied to selectable transform block 230 by encode/decode control line226 and forward/inverse transform control line 228.

The transformed data from selectable discrete cosine transform block 230is then applied to zig-zag quantization block 238 for quantization in aconventional zig-zag order. It will be understood that the need forintermediate storage is eliminated by quantizing in zig-zag order ratherthan first arranging the data into zig-zag order and then reading thearranged data into quantization block 238. This zig-zag quantization ofblock 238 is performed by indexed reading of the data from the matrix ofdata which is being quantized. This permits true pipelining withinaccelerator 120.

One basic formula for quantization by quantization block 238 withincompression/decompression accelerator 120 of the present invention maybe:

    value=d/(2×Q)                                        Equation (1)

where value is the quantized value which appears on quantization outputline 216 of quantization block 238, d is the input applied toquantization block 238 by selectable transform block 230, and Q is thequantization multiplier. Accelerator 120 may perform the divisionoperation of Equation (1) by performing a table look up and providing aneight bit value R which is given by the formula:

    R=256/(2×Q)

Compression/decompression accelerator 120 then performs the followingoperation:

    value=(d×R)>>8

If the coefficient is a DC term and the block is intrablock encoded:

    value=255 if (d is 1024)

    else value=d>>3.

The quantized output data from zig-zag quantization block 238 is appliedby way of quantization output line 216 to run length encoder block 246for conventional run length encoding within compression/decompressionaccelerator 120 in order to provide conventional run/value pairs. Therun/value output of run length encoder block 246 is applied to run/valuestore buffer 248 by way of encoder output line 242. Transmission ofrun/value pairs from run/value store buffer 248 to devices external toaccelerator 120 is by way of accelerator bus interface 200.

It will be understood that during the encoding process ofcompression/decompression accelerator 120 quantized data fromquantization block 238 may also be applied by way of quantization outputline 216 to dequantization multiplexer 218 and, therefrom, to zig-zagdequantization block 222. Thus quantization block 238 and dequantizationblock 222 are directly coupled within accelerator 120 by way ofdequantization output line 216. This direct coupling permitsdequantization immediately after quantization thereby avoiding thedelays of run length encoding and run length decoding. Thus pixelsapplied to accelerator 120 may be simultaneously encoded and decoded.

The data of line 216, which was zig-zag encoded in quantization block238, is applied to zig-zag dequantization block 222 in a sequenceadapted to reestablish the order of the data prior to the zig-zagencoding. This eliminates the need for the intermediate storage which isconventionally required to hold the re-ordered data. Thus, the datawhich is encoded in selectable discrete cosine transform block 230 andzig-zag encoded and quantized in quantization block 238 may beimmediately dequantized during the encoding process of accelerator 120by dequantization block 222 by way of quantizer output line 216. Thismay be performed without intermediate zig-zag and run length encoding ordecoding and without the memory storage requirements associatedtherewith.

The dequantization operation of dequantization block 222 may use theformula of Equations (2):

    d=Q×(2×value+sign(value))                      Equations (2)

    d=d-sign(d) if (d is even)

In this formula value is the quantized input data applied todequantization block 222 either by multiplexer input line 216 fromquantization block 238 or by multiplexer input line 211, Q is thequantization multiplier as previously described, and d is thedequantized result which appears on output line 223 of dequantizationblock 222.

In a case wherein the transform coefficient corresponds to a DC term andthe block being dequantized within dequantization block 222 isintrablock coded, the dequantization formula is as set forth inEquations (3). The intrablock coding decision is made within processorblock 112 and is communicated to dequantization block 202 by way ofINTRA control line 225.

    d=1024 if (value is 255)

    else d=8×value                                       Equations (3)

The data which is dequantized within dequantization block 222 is appliedto selectable discrete cosine transform block 230 by dequantizationoutput line 223. As previously described, selectable transform block 230is effective to selectively apply either a forward or a inverse discretecosine transform to received signals according to transform control line228. The inverse discrete transform is applied by selectable transformdevice 230 to the data received from unzig-zag dequantization block 222.

The inverse transformed signal provided by this operation is applied toframe add block 235 which also receives the reference frame data ofreference frame line 209. Frame add block 235 adds back in the referenceframe data value on line 209 which was subtracted out by framedifference block 220 during the encode process. The frame addition offrame add block 235 saturates its output to a value between zero andpositive two hundred fifty-five. The output of frame add block 234 isstored in decoded block storage buffer 240. Data which is thus encodedand decoded within accelerator 120 is transmitted from decoded blockstorage buffer 240 by way of accelerator bus interface 200. Thus, whileoperating in the encode mode compression/decompression accelerator 120provides both an encoded signal in buffer 248 and a decoded version ofthe encoded signal in buffer 240. This permits system 100 to determinethe errors which are decoded by a remote system receiving the encodeddata.

It will be understood that frame difference block 220 and frame addblock 235 may be reconfigurations of the same circuitry withinaccelerator 120. Add/subtract blocks 235, 220 may be selectably providedunder the control of control lines 226, 228 because they are not activeat the same time and there is enough latency to allow time toreconfigure the add/subtract circuit elements within them.

When encode/decode mode control line 228 adaptscompression/decompression accelerator 120 to operate in the decode mode,the data to be decoded is received from data bus 118 of remote videoprocessing system 100 into accelerator bus interface 200. This data tobe decoded in bus interface 200 has previously been discrete cosinetransformed, zig-zag encoded and run length encoded. It is thereforeapplied to run length decoder block 208 within accelerator 120. The runlength decoded output data of decoder 208 is applied by way of decoderoutput line 211 and dequantization multiplexer 218 to dequantizationblock 222 where it is reordered to undo the effects of zig-zag encodingas previously described. Control of dequantization multiplexer 218 maybe by forward/inverse transform control line 228 alone becausedequantization block 222 receives only the data of quantizer output line216 during forward transforms and only the data of decoder 208 duringinverse transforms.

In order to perform the zig-zag and the unzig-zag operations ofaccelerator 120, blocks 222, 238 translate the row order of thetransform coefficients, respectively, into the order of increasingspatial frequency. Rearranging the coefficients into this differentorder is a useful prelude to run length encoding because it tends togroup coefficients of similar spatial frequency together. The scanningorder of the coefficients used to apply data to quantization block 238of accelerator 120 is set forth in Table III wherein a zig-zag patternis followed from the upper left to the lower right. The run lengthencode process of encoder block 246 is a straight forward compression ofthe value string to a group of run/value pairs. The run length iscalculated by counting up to but not including the value.

                  TABLE III                                                       ______________________________________                                         1       2       6       7    15    16   28   29                               3       5       8      14    17    27   30   43                               4       9      13      18    26    31   42   44                              10      12      19      25    32    41   45   54                              11      20      24      33    40    46   53   55                              21      23      34      39    47    52   56   61                              22      35      38      48    52    57   60   62                              36      37      49      50    58    59   63   64                              ______________________________________                                    

Run length decoded data from run length decoder block 208 is applied byway of multiplexer input line 211 and dequantization multiplexer 218 todequantization block 222 where it is dequantized. Dequantizationmultiplexer 218 may be under the control of forward/inverse transformcontrol line 228 only as previously described. The dequantized data fromdequantization block 222 is then applied to selectable discrete cosinetransform block 230. Dequantization/quantization blocks 222, 238 areconfigured to perform the dequantization operation of block 222 underthe control of encode/decode control line 226.

It will be understood that dequantization multiplexer 218 is controlledto select multiplexer input line 211 for application to selectabletransform device 230 when compression/decompression accelerator 120 isoperating in the decode mode and to select quantization output line 216at the appropriate times when accelerator 120 is operating in the encodemode. Thus dequantization multiplexer 218 selects the externally encodeddata from bus interface 200 when in the decode mode and the internallyencoded data otherwise. This selection by multiplexer 218 is controlledby forward/reverse control line 228.

Thus, it will be understood that compression/decompression accelerator120 is a pipelined architecture which may be divided into two parallelpathways, one form encoding and one for decoding. The encoding pathwayreceives a previous image and a current image into buffers 204, 206respectively. The difference between the two may be applied by framedifference block 220 to selectable discrete cosine transform block 230.During this portion of the encode operation, discrete cosine transformdevice 230 operates in the forward transform mode. The remainder of thisencode pathway of accelerator 120 includes blocks 238, 246, whichreceive transformed data from transform device 230, perform furtherencoding operations, and store the encoded data in buffer 248.

When compression/decompression interface 120 operates in the decodemode, the other parallel pathway of its pipeline architecture is used.This other parallel pathway includes run length decoding anddequantization in blocks 208, 222 respectively. The dequantized data isapplied to discrete cosine block 230 which operates in the inversetransform mode when the decode pathway is active. The inverse transformdata is frame added and stored in buffer 240.

Finally, it will be understood that data may be applied from one pathwayto the other during the encode mode as previously described. This occurswhen quantized data on quantizer output line 216 of the encode parallelpathway is applied to dequantization block 222 in order to provide thecompanded image. Thus block 238 may be substantially directly coupled toblock 222 within accelerator 120. For example, blocks 222, 238 may workon the same block of physical memory within accelerator 120, with block222 working just a few pixels behind block 238.

Referring now to FIGS. 3A, B, there are shown encode dataflow 300 forperforming the encoding of data within compression/decompressionaccelerator 120 of the present invention and physical memory buffers350, 352, 354. Within encode dataflow 300 current image block 302 isapplied to motion estimation block 112a for a coding decision in orderto permit system 100 to transmit as little data as possible. For thispurpose, it will be understood that it is sometimes more efficient toestimate the displacement between one frame and the next and transmitonly the displacement. This is understood to be motion estimationencoding. However, if there is a great deal of difference betweenframes, it is less efficient to transmit the displacement and the blockis encoded based only upon itself. This is understood to be intrablockencoding. This determination, and therefore the determination whether toapply loop filter 210, is made by software coding decision block 112a.

Thus, within motion estimation block 112a a determination is madewhether current image 302 is to receive intrablock encoding or motionestimation encoding. Execution then proceeds tocompression/decompression accelerator 120 by way of line 312. If motionestimation is to be performed a pointer to the block to be encoded isstored in encode input circular buffer 322 which may be located inmemory 114. Thus, it will be understood that during encode dataflow 300pointers, rather than image data, reside in encode circular buffer 322and that dataflow 300 represents a loosely coupled system. Within encodeinput circular buffer 322 a pointer to current image block 322 is storedfor applying current image block 322 to compression/decompressionaccelerator 120 for motion estimation encoding.

Frame difference block 220 within accelerator 120 receives the previousblock and the data of current image block 326. A difference isdetermined within frame difference block 220 as previously described. Aforward discrete cosine transform is then performed on the data fromframe difference block 220 in encode dataflow 300 by forward discretecosine transform block 230a. It will be understood that the function offorward discrete cosine transform block 230a may be performed byselectable discrete cosine transform block 230 when selectable transformblock 230 operates in the forward mode as determined by transformcontrol line 228.

The transformed data from forward discrete cosine transform block 230ais received by quantization block 238 in a conventional zig-zag order aspreviously described and quantized therein. The quantized data fromblock 238 is applied by way of quantization output line 216 to runlength encoder 246 for run length encoding. Run length encoding withinencoder 246 provides conventional run/value pairs as known to thoseskilled in the art. The run/value pairs from run length encoder 246 areapplied, by way of line 330, to encode output circular buffer 332.

The data within encode output circular buffer 332 is then applied tovariable length encoder 112b to provide compressed bit stream 338. Itwill be understood that buffer 332 may be located in memory 114 and thatvideo processor 112 may read the run/value pairs from memory 114 inorder to perform the operations of variable length encoder 112b. Thus,the motion estimation decision of motion estimation block 112a and thevariable length encode of variable length encoder 112b may both beperformed by video processor 112 of system 100. A video processor systemsuitable for this purpose is the 82750 PB made by Intel Corporation.

It will thus be understood that the operations of both motion estimationblock 112a and variable length encode 112b may be performed by thisvideo processor system. Thus the functions of encode dataflow 300 arepartitioned as follows: (1) the software functions including motionestimation and Huffman encoding are performed by video processor 112,and (2) the remaining functions, in particular functions which would becomputationally intensive are hardwired in circuitry within accelerator120. These partitioned functions of processor 112 and accelerator 120are synchronized within remote video processing system 100.

The quantized data from quantization block 238 is also applied todequantization block 222 by way of quantization output line 216. Thedata transferred in this manner within compression/decompressionaccelerator 120 thus bypasses both run length encoding within run lengthencoder block 246 and run length decoding within run length decoderblock 208. Thus, the process of decoding the encoded data in order toprovide a companded image may be performed without the additional memoryreads and writes involved in performing run length encoding and runlength decoding. These savings provide a significant advantage in realtime calculations. It will be understood by those skilled in the artthat the operations of run length blocks 208, 246 are not lossyoperations and that their contribution to the motion estimation decodingdecision of motion estimation block 112a may therefore be ignored.

Inverse discrete cosine transform block 230b applies an inversetransform to the dequantized data received from dequantization block222. It will be understood that the operations of inverse discretecosine transform block 230b of encode dataflow 300 may be performed byselectable discrete cosine transform block 230 when transform block 230operates in the inverse mode under the control of transform control line228. Thus during the encode mode of compression/decompressionaccelerator 120, as described by encode dataflow 300, selectabletransform block 230 functions both as a forward and an inverse discretecosine transform under the control of forward/inverse transform controlline 228.

The decoded signal from inverse transform block 230b is applied to frameaddition block 235, and, therefrom, to current/companded image block 302by way of line 328. It will be understood that the signal of line 328resulting from encoding and decoding within encode dataflow 300 ofaccelerator 120 is the image which is compared with the previous imageof previous image block 314 in order to make the motion estimationcoding decision of block 112a. A useful and advantageous feature ofencode dataflow 300 is that current image 326 is overwritten by thedecoded image of line 328 thereby eliminating the need for separatebuffers storing the same images. Thus portions of the companded imageand portions of the current image are present simultaneously in the sameblock of memory which may thus simultaneously perform the functions ofboth blocks 302, 326.

The operation of blocks 302, 326 may be better understood by referenceto physical buffers memory 350, 352, 354. During operations of remotevideo processing system 100 digitized image data from camera 127 may bestored first in physical buffer memory 350 while physical buffer memory352 stores the previous image. Physical buffer memory 354 stores portion358 of the current image which is in the process of being overwritten bythe companded portion 356.

When an entire digitized image is stored in buffer memory 350 and anentire companded image is stored in physical memory 354, the functionsof physical buffer memories are redefined. The digitized data inphysical memory 350 is used as the current image 362 which isoverwritten by companded data 360. The companded data of physical memory354 is used as the previous image for motion estimation by block 112a.New digitized data is stored in physical memory 352. This processcontinues with physical memory 354 next serving to collect new digitizeddata. In this manner three, rather than four blocks of physical memorymay be used to perform these operations of encode dataflow 300.

Referring now to FIG. 4, there is shown encode input buffer datastructure 400 for controlling blocks of data stored in encode inputcircular buffer 322 as required within encode dataflow 300. Withinencode input buffer data structure 400 control word 402 is provided.When motion estimation coding decision block 112a determines how animage should be encoded a bit within control word 402 may be used toindicate whether the block is intrablock encoded or a motioncompensation encoded. Other parameters associated with each block ofdata may also be stored in control word 402 of data structure 400.Another example of the type of information which may be stored withincontrol word 402 is information on whether selectable loop filter 210 isenabled or disabled. Additionally the five-bitquantization/dequantization value Q of blocks 222, 238 may be stored incontrol word 402 of encode input buffer data structure 400.

Encode input buffer data structure 400 also contains original imageblock address pointer 404. Original image block address pointer 404points to the original location of a block of data in current imageblock 326. It will be understood that original image block pointer 404also points to current/companded image block 302 when image blocks 302,326 are in the same physical memory because the companded image receivedby way of line 328 overwrites the current image. Thus the image beingencoded is the original image which may be the same as the currentimage. It will also be understood that pointer 404 points to fixedlength data rather than variable length data because the image of block302 is not yet encoded. Previous image block address pointer 406 pointsto the location of the previous image in block 314.

Referring now to FIG. 5, there is shown encode output buffer datastructure 500 for encode output circular buffer 322 within encodedataflow 300. In the preferred embodiment of compression/decompressionaccelerator 120 two input run/value pairs such as run/value pairs 501,503 may be stored in a thirty-two bit double word of input circularbuffer 322, such as double words 502, 504. Double word 506 of inputcircular buffer 322 is filled with ones to indicate the end of a list ofrun/value pairs. When there is an odd number of run/value pairs in thelist of buffer data structure 500 one word of double word 506 may beused to store a run/value pair and the remaining word may be filled withones to indicate the end of the list.

Referring now to FIG. 6, there is shown decode dataflow 600 usingcompression/decompression accelerator 120 of the system of the presentinvention. Compressed bit stream 601 is received and decompressed withindecode dataflow 600. Compressed bit stream 601 may be a bit stream suchas compressed bit stream 338 provided by encode dataflow 300 of thepresent invention. A variable length decode is performed upon receivedbitstream 338 by variable length decoder 112c. It will be understoodthat variable length decoder 112c of decode dataflow 600 and variableencoder 112a of encode dataflow 300 may be performed by a single dualpurpose encoding and decoding device within video processing system 100such as processor block 112. The decoded data of variable length decoder602 is then placed into decode input circular buffer 606 of decodedataflow 600.

Frame difference block 220, forward discrete cosine transform 230a,quantization block 246 and run length encoder 246 are not used withincompression/decompression accelerator 120 when acceleration 120 operatesin the decode mode. When operating in the decode mode accelerator 120receives the data for decoding according to decode dataflow 600 fromdecode input circular buffer 606. In particular, run length decoder 208of accelerator 120 receives the data from circular buffer 606. The datadecoded within run length decoder 208 is then applied to dequantizationblock 222 for a dequantization within accelerator 120. This data isapplied to dequantization block 222 in an order which is adapted torestore the sequence of the data prior to zig-zag encoding by a zig-zagencoder within an external device which transmits encoded data toaccelerator 120.

An inverse discrete cosine transform is applied to the dequantized dataof block 222 by inverse discrete cosine transform block 230b. It will beunderstood that the operations of inverse discrete cosine transformblock 230b within decode dataflow 600 may be performed by selectabletransform block 230 when selectable transform block 230 is in theinverse mode as determined by transform control line 228.

Previous image block 604 is received by compression/decompressionaccelerator 120 by way of line 203. The received previous image block604 is stored in previous block memory 206 within accelerator 120. Ifselectable loop filter 210 is enabled under the control of filtercontrol line 214 the image in previous block memory 206 is filtered andapplied to frame add block 234. If selectable loop filter 210 is notenabled the image within previous block memory 206 is applied directlyto frame add block 234. When the previous image block 604, as receivedfrom previous block memory 206, and the dequantized transformed currentimage block are added in frame add block 234, the output of block 234 isstored in decoded image block 608.

Referring now to FIG. 7, there is shown decode input circular bufferdata structure 700 for decode input circular buffer 606 of decodedataflow 600. It will be understood that the data of decode inputcircular buffer 606 is variable length data if Huffman encoding isperformed before transmission to buffer 606. Decode input buffer datastructure 700 contains control word 702 which may be used to store thequantization/dequantization value Q as well as information regardingwhether selectable loop filter 210 is enabled or disabled. Additionallycontrol word 702 may contain information regarding whether the imageblock being decoded by decode dataflow 600 is a intracoded block or amotion compensated block. Current image block address 704 of datastructure 700 points to the beginning of a current image and previousimage block 706 points to the previous image.

Additionally, in the preferred embodiment of data structure 700 tworun/value pairs, such as run/value pairs 710, 712, may be stored in eachof a number of double words such as double words 708, 710. Double word716 of circular buffer 606 is filled with ones to indicate the end ofthe list of run/value pairs. If there is an odd number of run/valuepairs in the list of pairs, one word of double word 716 may be used tostore a run/value pair and the remaining word may be filled with ones toindicate the end of the list.

It will be understood that motion estimation coding decision block 112aperforms the motion estimation process within encode dataflow 300.Additionally, it performs the variable length encode of the local imageand the variable length decode of remote compressed bit stream 601.Thus, the functions of block 112a may be performed by a general purposevideo processor such as video processor 112. Compression/decompressionaccelerator 120 of the present invention performs all of the remainingfunctions of encode dataflow 300 and decode dataflow 600.

Video processor 112 and accelerator 120 of remote video processingsystem 100 operate substantially independently of each other. However,they tend to work from common areas of memory to implement operationssuch as accesses to the various circular buffers. The manner in whichthe work load of remote video processing system 100 is partitionedbetween video processor 112 and compression/decompression accelerator120 minimizes the overhead required for synchronization of the variousdataflows such as dataflows 300, 600. Nevertheless, some possibilitiesfor conflict still exist within video processing system 100. Thus, theremust be a method within video processing system 100 to manage memorysuch that buffer overflows and underflows due to conflicts between videoprocessor 112 and accelerator 120 are avoided.

In order to minimize costly fine tuning of synchronization within remotevideo interface system 100 a pointer interlock scheme is used forreading and writing the run/value data in encode dataflow 300 and decodedataflow 600. To implement this scheme both an encode synchronizationand a decode synchronization are provided within remote video processingsystem 100 containing compression/decompression accelerator 120.

Referring now to FIGS. 8, 9, there are shown input circular buffersynchronization flow 800 and output circular buffer synchronization flow900, as well as various buffer pointers for use in circular buffersynchronization flows 800, 900. It will be understood that videoprocessor 112 of remote video interface system 100 controlssynchronization of access to buffers 322, 332 within remote videoprocessing system 100 buffers in accordance with input/outputsynchronization flows 800, 900. During the encoding process of encodedataflow 300 compression/decompression accelerator 120 reads block listsof uncompressed image data from encode input circular buffer 322.Accelerator 120 also writes compressed data in the form of run/valuepairs into encode output circular buffer 332 within dataflow 300 aspreviously described. Access by accelerator 120 to the physical memorystoring circular buffers 322, 332 must be synchronized in order to avoidconflicts.

Therefore, in order to avoid conflicts, whenever a reset or some otherinitial condition occurs within video processor 112 andcompression/decompression accelerator 120 first the pointers of one pairof input buffer pointers are set equal to each other and then thepointers of a second pair of output buffer pointers are set equal toeach other. Accelerator INRPTR pointer 804b, which indicates the nextlocation from which accelerator 120 will read input circular buffer 322,is set equal to accelerator INEPTR pointer 804c, which indicates thenext location after end location 804 of current input list 812.

Compression/decompression accelerator 120 then sets the two outputbuffer pointers equal to each other. Accelerator OTWPTR pointer 830c,which indicates the next location to which accelerator 120 will write inoutput circular buffer 322, is set equal to the value of acceleratorOTEPTR pointer 830e, which indicates the location after the end of thecurrent output list.

The beginning and the end of both the current input list are thus equalto each other and the beginning and end of the current output list arethus equal to each other. This defines their stall condition. Therefore,processing by compression/decompression accelerator 120 within encodedataflow 300 is suspended. It should be noted, however, that businterface 200 of accelerator 120 may continue to operate becauseaccelerator 120 may be a target in a bus transaction of remote videointerface system 100. However, accelerator 120 does not initiate anytransactions in this suspended state.

After the above pairs of buffer printers are set equal to each other,video processor 112 defines the areas of encode input circular buffer322 and encode output circular buffer 332. These buffers are defined bysetting a series of memory mapped pointers withincompression/decompression accelerator 120 using scalar memory writeoperations. These pointers include accelerator INCSTR pointer 804a andaccelerator INCEND pointer 804d which indicate the location afterbeginning address 803 and the location after ending address 805,respectively, of encode input circular buffer 322. Additionally,accelerator pointers OTCSTR 840a, OTCEND 840d are set by video processor112.

It will be understood that video processor 112 must set acceleratorpointers 804a, c, d equal to processor pointers 820a, c, d so thataccelerator 120 and block 112 may agree regarding where circular buffer322 begins and ends and where current list 812 ends. Thus, for example,after video processor 112 writes one or more blocks of data, and updatesits own write pointer 820e in a post increment manner, it updatesaccelerator pointer INCEND 804d.

Processor INWPTR pointer 820e indicates location 806 within buffer 322.Location 806 is the location to which video processor 112 is writing andit has no corresponding pointer within compression/decompressionaccelerator 120 because accelerator 120 does not write to buffer 322.Accelerator INRPTR 804b and processor INRPTR 820b within video processor112 indicate the next location at which accelerator 120 will read inputcircular buffer 322 and thus will differ from each other.

Processor block 112 then starts motion estimation and blockclassification on each block of pixels in accordance with thedeterminations previously described with respect to encode dataflow 300.Video processor 112 then writes the pointers associated with each blockof externally stored image data into input circular buffer 322, startingwith the address indicated by INWPTR pointer 820e.

In the case of a decode of a remote bitstream 601,compression/decompression accelerator 120 reads input data from decodeinput circular buffer 606 and writes reconstructed data into a framememory such as decoded image block 608. Under initial conditions,including a reset, accelerator 120 sets two input buffer pointers equalto each other. Accelerator INRPTR pointer 804b, which indicates theposition from which accelerator 120 is reading input circular buffer606, is set equal to accelerator INEPTR pointer 804c, which representsthe end of the current input list.

Video processor 112 then defines the areas of encode input circularbuffer 606 and encode output circular buffer 608 by setting a series ofmemory mapped pointers on compression/decompression accelerator 120using scalar memory write operations. This may be the only time whenvideo processor 112 writes to accelerator 120. These pointers includeINCSTR pointer 804a and INCEND pointer 804d which indicate the beginningand ending addresses, respectively, of encode input circular buffer 322.Video processor 112 then starts the variable length decode process onthe compressed data and writes the run/value block data to the circularbuffer area.

When enough data is written into the buffer area, processor 112 updatesboth INEPTR pointers 804c, 820c in compression/decompression accelerator120 and in processor 112. INEPTR pointers 804c, 820c point to thelocation after the last location of the data in the buffer area. Ingeneral the flow pointers of synchronization flows 800, 900 areincremented after a read or a write and thus point to the next locationto be read or written. Compression/decompression accelerator 120 thenbegins to fetch data at the location programmed into accelerator INRPTRpointer 804b and continues up to but not including the locationprogrammed into accelerator INEPTR pointer 804c. When INRPTR pointer804b equals INEPTR pointer 804c, compression/decompression accelerator120 stops processing.

INRPTR pointer 804b wraps around to pointer INCSTR 804a when it reachesthe address programmed into accelerator INCEND pointer 804d. Videoprocessor 112 is responsible for maintaining both INEPTR pointers 804c,820c. Compression/decompression accelerator 120 is responsible formaintaining accelerator INRPTR pointer 804b. Video processor 112 updatesits copy of INRPTR pointer 820b by reading accelerator INRPTR pointer804b within compression/decompression accelerator 120.

Video processor 112 may write additional blocks into input circularbuffer 322 by writing the data then updating accelerator INEPTR pointer804c. When video processor 112 writes this data, accelerator INRPTRpointer 804c is also read. This is required in order to prevent videoprocessor 112 from overwriting areas of the circular buffer. Thisdefines the stall condition for a device writing into the circularbuffer when its read and write pointers are the same. It will beunderstood that it is also the responsibility of video processor 112 toupdate the pointers of accelerator 120 during output to output circularbuffer 332. Accelerator 120 updates its own end pointer and writepointer and video processor 112 must read the updated pointers.

Referring now to FIGS. 10, 11, there are shown forward discrete cosinetransform computation flow 1000 and inverse discrete cosine transformcomputation flow 1100 of the present invention. Discrete cosinetransform computation flows 1000, 1100 may be performed by transformblocks 230a, b respectively of dataflows 300, 600. Additionally discretecosine transform computation flows 1000, 1100 may be performed byselectable transform block 230 under the control of forward/inversetransform control line 228.

The operation of forward discrete cosine transform computation flow 1000as performed by transform block 230a of encode dataflow 300 orselectable discrete cosine transform block 230 ofcompression/decompression accelerator 120 may be expressed as shown inEquation (4):

    [Φ]=[C][X].                                            Equation (4)

In Equation (4) matrix [X] is the input data matrix applied to transformblock 230 or transform block 230a, matrix [C] is the discrete cosinetransform matrix, and matrix [Φ] is the transformed output matrix whichis applied to transform output line 236 by transform block 230.

Inverse discrete cosine transform computation flow 1100 as performed bytransform block 230 or by selectable discrete cosine transform block 230under the control of transform control line 228 may be expressed shownin Equation (5):

    [X]=[C].sup.T [Φ].                                     Equation (5)

In Equation (5) matrix [Φ] is the transformed input matrix received byway of transform input line 223, matrix [X] is the output matrix appliedto transform output line 234, and matrix [C]^(T) is the inverse discretecosine transform matrix.

The individual coefficients c_(m) of the discrete cosine transformmatrix [C] may be expressed as: ##EQU2## In Equation (6) N is the orderof the discrete cosine transform performed within transform computationflows 1000, 1100 and m and n are the row and column indices,respectively, of the discrete cosine transform matrix [C] wherein m andn have the values 0, 1, 2, . . . , N-1. The constant k_(m) has a valueof one divided by the square root of two if the row index m has a valueof zero. The constant k_(m) has the value of one if the row index m isnot zero.

Solving Equation (6) when the order N of the discrete cosine transformis eight yields the following discrete cosine transform coefficientsc_(m) : ##EQU3## Under these circumstances the discrete cosine transformmatrix [C] of Equation (4) may be formed in accordance with Equation (6)and Equations (7) as follows: ##STR1##

It will be understood that eight multiply/accumulate operations arerequired to perform this transform for each data point within input datamatrix [X]. Therefore 64×8=512 multiply/accumulate operations arerequired for a one-dimensional discrete cosine transform. For atwo-dimensional discrete cosine transform 1024 multiply/accumulateoperations are required.

Several fast prior art algorithms are known for performing the forwardand inverse discrete cosine transforms of Equation (4) and Equation (5).Using certain of these prior art methods an eight point discrete cosinetransform may be performed with twelve multiplies and twenty-nine adds.Thus a total of one hundred ninety-two multiplications and four hundredsixty-four additions are required thereby simplifying the operation ofthese transforms. These algorithms are better suited for paralleloperation.

Forward discrete cosine transform flow 1000 of the present inventionexecutes a fast forward discrete cosine which is a faster and moreefficient variation of the transform represented by Equation (4) . Inforward discrete cosine transform flow 1000, the order N of thetransform is eight. It will be understood by those skilled in the artthat the transform performed by forward transform flow 1000 is a fastforward transform of the type described with respect to transform block230a. It is performed by selectable discrete cosine transform block 230when selectable transform block 230 is in the encode mode. This fastforward transform may be expressed as: ##EQU4## wherein the elements ofsubmatrices [C_(A) ] and [C_(B) ] are obtained from the discrete cosinetransform matrix using sparse matrix factorization techniques known tothose skilled in the art and the coefficients of Equations (8) are givenby: ##EQU5## The elements of vectors [X^(') ] and [X^(") ] orsubmatrices [X^(') ] and [X^(") ] are formed by respectively adding andsubtracting the high order data points of matrix [X] and the low orderdata points of matrix [X].

Transform computation flow 1000, performed by selectable discrete cosinetransform block 230, is effective to receive the input data matrix [X]and apply the forward discrete cosine transform matrix [C] to input datamatrix [X] to provide the forward transformed matrix[Φ] as set forth inEquation (4) and Equations (8). In order to perform these operations thelow order data points x₀ -x₃ of an input word x are selected to form asubword. These data points may be any number of bits wide. The datapoints of this subword are placed into circular input registers 1006.Similarly, the high order data points x₄ -x₇ of the input word areselected to form another subword. The data points of this subword areplaced into circular input registers 1032.

As data points x₀ -x₃ are successively applied to addition node 1008 andsubtraction node 1036 they are also applied to the input of circularregisters 1006 by way of a loop formed by register output line 1004.Similarly, data points x₄ -x₇ are successively applied to the input ofcircular registers 1032 by way of a loop formed by register output line1030 as they are applied to addition node 1008 and subtraction node1036.

The timing of the presentation of each of these data points iscontrolled in a manner understood by those skilled in the art to providethe sums x₀ +x₇, x₁ +x₆, x₂ +x₅, and x₃ +x₄, which are the elements ofsubmatrix [X^(') ] of Equations (8), at the output of addition node1008. In a similar manner the differences x₀ -x₇, x₁ -x₆, x₂ -x₅, x₃-x₄, which are the elements of submatrix [X^(") ] of Equations (8), areformed at the output of subtraction node 1036. It will be understoodthat, acting cooperatively, register output lines 1004, 1030 and nodes1008, 1036 operate as a conventional butterfly adder operating upon aseries of pairs of input data points. In a conventional butterfly adder,two inputs are received and two outputs are provided, one output beingthe sum of the inputs, the other the difference. In the case of flow1000, these inputs proceed through input circular buffers 1004, 1032 toproduce the values of submatrices [X^(') ] and [X^("]). The output ofnodes 1008, 1036 are then successively applied to multiplication nodes1014, 1040 within transform computation flow 1000.

The coefficients of submatrix [C_(A) ] are applied to submatrix [X^(') ]received by multiplication node 1014 from addition node 1008 formultiplication within node 1014 in accordance with Equations (8). Thecoefficients of the submatrix [C_(A) ] are applied to multiplicationnode 1014 by coefficient register 1010. The matrix partial product termsthus formed by multiplication node 1014 are then applied to additionnode 1016 within transform computation flow 1000.

In a similar manner submatrix [X^(") ] received by multiplication node1040 from subtraction node 1036 is multiplied within node 1040 by thecoefficients of submatrix [C_(B) ] in accordance with Equations (8). Thecoefficients of submatrix [C_(B) ] are applied to multiplication node1040 by coefficient register 1042. The matrix partial product terms thusformed by multiplication node 1040 are applied to addition node 1044within transform computation flow 1000.

The sums formed by addition nodes 1016, 1044 are applied to registerblocks 1018, 1046, respectively, within forward discrete cosinetransform computation flow 1000. The outputs of register blocks 1018,1046 are applied back to addition nodes 1016, 1044, respectively, by wayof register output lines 1020, 1047 causing delayed terms to be summedwith terms nearby received by nodes 1016, 1044. This permits transformcomputation flow 1000 to perform the additions of the partial productterms as required by the matrix algebra operations of Equations (8).

The output of register blocks 1018, 1046 are also applied to registerblocks 1024, 1050, respectively, within forward transform computationflow 1000 which may be implemented by selectable discrete cosinetransform block 230. It will be understood that the output of registerblock 1024, which appears on register output line 1026, is the set ofeven numbered transformed data points φ₆, φ₄, φ₂, φ₀ of Equations (8).Additionally, it will be understood that the output of register block1050, which appears on register output line 1052 is the set of oddnumbered transformed data points φ₇, φ₅, φ₃, φ₁ of Equations (8). Thevalues of register output lines 1026, 1052 are applied to flow outputmultiplexer 1054 in order to be multiplexed, reordered and applied byway of the forward flow output line 236. It is believed that performinga fast forward discrete cosine transform using the system and method ofcomputation flow 1000 provides a transformed output signal several clockcycles faster than the known prior act.

In this manner the transform of computation flow 1000 may be performedby applying a single multiplier 1014, 1040 to each of the outputs ofcircular buffers 1006, 1032 as the data circulates from one end into theother by way of the loops of lines 1004, 1030. It will be understoodthat the data must circulate through circular buffers 1006, 1030 oncefor each pair of output data points from buffers 1006, 1032.

In a similar manner inverse discrete cosine transform computation flow1100 of the present invention executes a fast inverse discrete cosinetransform wherein the order N of the transform is eight. It will beunderstood by those skilled in the art that the transform performed byforward transform computation flow 1100 is the type of transformperformed by selectable discrete cosine transform block 230 whentransform block 230 is in the decode mode. This inverse discrete cosinetransform matrix [C]^(T) may be mathematically expressed as: ##EQU6##wherein the submatrices [C_(A) ]^(T) and [C_(B) ]^(T) are obtained fromthe discrete cosine transform matrix using sparse matrix factorizationtechniques.

Computation flow 1100 of selectable discrete cosine transform block 230is effective to receive an inverse transformed matrix [φ] and apply aninverse discrete cosine transform matrix [C]^(T) to provide an outputdata matrix [X]. Computation flow 1100 is adapted to be a much fastervariation of the transform set forth in Equation (4) and Equations (9).In order to perform these operations the even transformed data pointsφ₀, φ₂, φ₄, φ₆, of the values of transformed matrix [φ] are placed incircular input registers 1106 and the odd transformed data points φ₁,φ₃, φ₅, φ₇ reside in circular input registers 1132.

As even transformed data points φ₀, φ₂, φ₄, φ₆ residing in circularinput registers 1104 are successively applied to multiplication node1114 they are simultaneously applied to the input of circular registers1106 by register output line 1104. As odd numbered transformed datapoints φ₁, φ₃, φ₅, φ₇, of circular input registers 1132 are applied tomultiplication node 1140 they are also simultaneously applied back tothe input of circular registers 1130 by way of a loop formed by registeroutput line 1130.

The value applied to multiplication node 1114 from input registers 1106is multiplied within multiplication node 1114 by the coefficients ofsubmatrix [C_(A) ]^(T) in accordance with Equations (8). Thecoefficients submatrix [C_(A) ]^(T) are applied to multiplication node1114 by coefficient register 1110. The product formed by multiplicationnode 1114 is then applied to addition node 1116.

In a similar manner the value received by multiplication node 1140 frominput registers 1130 is multiplied within node 1140 by the coefficientsof submatrix [C_(B) ]^(T) in accordance with Equations (8). Thecoefficients of submatrix [C_(B) ]^(T) are applied to multiplicationnode 1140 from coefficient register 1142. The product thus formed bymultiplication node 1140 is applied to addition node 1144.

The sums formed by addition nodes 1116, 1144 are applied to registerblocks 1118, 1146, respectively, within inverse discrete cosinetransform computation flow 1100. The outputs of register blocks 1118,1146 are applied back to addition nodes 1116, 1144 by way of registeroutput lines 1120, 1147. In this manner, the various partial productterms of Equations (9) may be provided within transform computation flow1100.

The output of register blocks 1118, 1146 are also applied to bothaddition node 1122 and subtraction node 1148 by way of register outputlines 1120, 1147. In this manner the combinations of the partial productterms required by the matrix algebra of Equations (9) may be performed.The outputs of addition node 1122 and subtraction node 1148 are thenapplied to register blocks 1124, 1150, respectively, within inversetransform computation flow 1100 which may be implemented by selectablediscrete cosine transform block 230.

It will be understood that the output of register block 1124, whichappears on register output line 1126, includes the low order bits x₀,x₁, x₂, x₃ of Equations (9). Additionally, it will be understood thatthe output of register block 1150, which appears on line 1152, includesthe high order data points x₄, x₅, x₆, x₇ of Equations (9). The valuesof lines 1126, 1152 are applied to output multiplexer 1154 in order tobe multiplexed and reordered to be applied to frame add 235 by way ofoutput line 234.

Referring now to FIG. 12, there is shown a block diagram representationof selectable discrete cosine transform block 230 ofcompression/decompression accelerator 120. It will be understood thatselectable discrete cosine transform block 230 is effective to performthe operations of both forward discrete cosine transform computationalflow 1000 and inverse discrete cosine transform computational flow 1100depending upon the control signal applied by way of transform controlline 228.

When compression/decompression accelerator 120 performs the operationsof encode dataflow 300, selectable transform block 230 performs both theoperations of forward transform block 230a and inverse transform block230b. Which transform is performed depends on whether the data ofmultiplexer 224 is being encoded or the data of dequantizer 222 is beingdecoded. It is the operations upon the data of multiplexer 224 which arerepresented by forward discrete cosine transform computational flow1000. When performing the operations of encode dataflow 300 relevant toinverse block 230b compression/decompression accelerator 120 provideboth an encoded data stream and an image representative of the decodingof the encoded data stream. When accelerator 120 performs the operationsof decode dataflow 600, selectable discrete cosine transform block 230performs the operations of inverse transform block 230b as describedwith respect to inverse computational flow 1100. These selectableforward and inverse transform operations are determined by the signalapplied to selectable transform block 230 by way of transform controlline 228 as previously described.

Within selectable discrete cosine transform block 230 data from blockmemory 1242 is applied by way of data bus 1238 to input databuffer/shifter 1212. Input data buffer/shifter 1212 is effective toarrange and order input data either as shown with respect to inputcircular registers 1006, 1032 during a forward transform or as shownwith respect to input circular buffers 1106, 1132 during an inversetransform. This arranging and ordering of the input data thus providesinput matrices [X^(') ] and [X^(") ] of Equations (8) or input matrices[Φ^(') ] and [Φ^(") ] of Equations (9) depending upon whether a forwardtransform or an inverse transform is performed by selectable discretecosine transform block 230.

The correctly ordered output of input data buffer/shifter 1212 isapplied by way of line 1214 to multiplexers 1208, 1234. Multiplexers1208, 1234 are controlled by transform control line 228 which selectsthe encode and decode modes of selectable transform block 230 aspreviously described. The signal of transform control line 228 isdetermined according to control logic 1240. Control logic 1240 providesa logic level representative of the operating mode ofcompression/decompression accelerator 120 according to determinationsmade by processor block 112 in a manner understood by those skilled inthe art.

When selectable transform block 230 is in the inverse mode, transformcontrol line 228 causes array input multiplexer 1208 to select the dataof multiplexer line 1214 and apply the data of line 1214 tomultiplier/accumulator array 1210. Thus the input data frombuffer/shifter 1212 is applied substantially directly tomultiplexer/accumulator array 1210 when selectable transform block 230is in the decode mode. The coefficient matrices [C_(A) ]^(T) and [C_(B)]^(T) are then applied to array 1210 from coefficient registers 1204 inorder that array 1210 may perform the operations of Equations (9) aspreviously described with respect to inverse computation flow 1100.Transformed data in output data buffer 1230 may be transmitted to blockmemory 1242 by way of bus 1238.

When selectable discrete cosine transform block 230 is in the encodemode, the data selected by array input multiplexer 1208 to be applied tomultiplier/accumulator array 1210 is the data on multiplexer input line1220 rather than the data on multiplexer input line 1214. The data onmultiplexer input line 1220 is the output of butterfly adder 1218 oradd/subtract device 1218. Butterfly adder receives as its input either(1) the data from input data buffer/shifter 1212 by way of line 1214, or(2) the output of multiplier/accumulator array 1210. Which of these twodata streams is applied to butterfly adder 1218 is determined bymultiplexer 1234 under the control of transform control line 228.

Thus the operations performed upon input data by selectable transformblock 230 prior to multiplication by the transform coefficients, aspreviously described with respect to addition node 1008 and subtractionnode 1036, may be performed within selectable transform block 230 whenselectable transform block 230 is in the forward mode. This alsoperforms the operations of the feedback loops formed by register outputlines 1020, 1046 as well as addition nodes 1016, 1044.

It will therefore be seen that the arithmetic operations of butterflyadder 1218 may be applied directly to the input data points provided bybuffer/shifter 1212. Alternatively, adder 1218 may be applied to theinput data points provided by buffer/shifter 1212 after the input hasbeen operated upon by the transform coefficients of array 1210. Which ofthese configurations is provided depends on whether selectable transformblock 230 is in the mode for performing a first discrete cosinetransform or the inverse of the first discrete cosine transform. It willalso be seen that in either mode all functional elements withintransform block 230 are used and there is thus no wasted hardware.

For example, the same hardware elements within transform block 230 mayserve both as registers 1006 and as registers 1106, as registers 1032and registers 1132. These registers together may form buffer 1212.Similarly, the same hardware elements within transform block 230 mayserve both as adder 1008 and as adder 1122, as subtractor 1036 andsubtractor 1148. These arithmetic elements operating cooperatively mayserve as butterfly adder 1218. Likewise coefficient matrices [C_(A) ]and [C_(A) ]^(T) may be the same hardware circuitry which is merelyaccessed differently depending on the mode of selectable transform block230.

Thus selectable transform block 230 requires two multipliers and fouradders to perform either the forward or inverse discrete cosinetransform. Either of these transforms can be performed by transformblock 230 in sixteen clock cycles. It will be understood that this timemay be cut in half by providing two multipliers to operate upon theoutputs of each buffer 1106, 1132 or each buffer 1006, 1032. However thesize of the resulting transform circuit would be twice as large.

Referring now to FIG. 13, there is shown stepping direction chart 1300.Stepping direction chart 1300 represents a center position P_(c)surrounded by four positions P₁ -P₄ along with a plurality of steppingdirections 1302-1316. Stepping directions 1302-1316 represent thedirections that positions P_(c), P_(1-P) ₄ may move from one frame toanother frame during a display of remote video processor system 100. Forexample, if positions P_(c), P₁ -P₄ of stepping direction chart 1300move directly upwards from one frame to the next, stepping direction1302 represents their displacement. If positions P_(c), P₁ -P₄ movedirectly to the right, their motion is represented by steppingdirections 1306. Stepping direction 1304 represents the motion ofposition P_(c), P₁ -P₄ when the positions of stepping direction chart1300 move to the upper right. In the manner, eight different directionsare represented by stepping directions 1302-1316 of stepping directionchart 1300.

Referring now to FIGS. 14A-C, there is shown a flow chart representationof motion estimation method 1400. Also show is a representation ofmotion estimation method 1440 which is an alternate embodiment of motionestimation method 1400. The function of motion estimation methods 1400,1440 is finding the best match for a target region during apredetermined period of time. Motion estimation methods 1400, 1440 maybe applied to positions P_(c), P₁ -P₄ of motion vector chart 1300 todetermine which stepping direction 1302-1316 best represents the motionof positions P_(c), P₁ -P₄ from one frame to another. It will beunderstood that both motion estimation methods 1400, 1440 may be used inencoding dataflow 300 of remote video processing system 100 of thepresent invention.

Operation of motion estimation method 1400 begins with a determinationin decision 1402 whether center position P_(c) or position P₁ of thecurrent frame is a better match with the center position of the previousframe. This determination, as well as the determination of each of theremaining candidate positions tested in method 1400, requires acomparison of two hundred fifty-six pixels values of asixteen-by-sixteen pixel block in the preferred embodiment of encodedataflow 300. If position P₁ is a better match than center positionP_(c) the best horizontal position P_(H) is determined to be position P₁as shown in block 1404 by motion estimation method 1400.

If position P₁ is not a better match than position P_(c) a determinationis made in decision 1406 whether position P₂ is a better match with thecenter position of the previous frame than center position P_(c) of thecurrent frame. If position P₂ is a better match than center positionP_(c) the best horizontal position P_(H) is made equal to position P₂ asshown in block 1408. If neither position P₁ nor position P₂ is a bettermatch than center position P_(c) then center position P_(c) is selectedas the best horizontal position V_(H) as shown in block 1410. Thus, whenoperation of motion estimation method 1400 arrives at point 1411, adetermination has been made which of the three horizontal positionsP_(c), P₁, P₂ of the current frame has the greatest improvement from thecenter position. The best horizontal position P_(H) is the onedetermined to be the best match.

A determination is then made within motion estimation method 1400whether position P₃ is a better match than center position P_(c) indecision 1412. If position P₃ is a better match it is selected as thebest vertical position P_(V) as shown in block 1414. If position P₃ isnot a better match than center position P_(c) a determination is made indecision 1416 whether position P₄ is a better match than center positionP_(c). If position P₄ is a better match, as determined in decision 1416,it is selected as the best vertical position P_(v) as shown in block1418. If neither position P₃ nor position P₄ 1416, is a better matchthan center position P_(c), as determined in decisions 1412, centerposition P_(c) is selected as the best vertical position P_(v) as shownin block 1420. Thus, when operation of motion estimation method 1400arrives at point 1421 the best vertical position P_(v) has been setequal to either center position P_(c), position P₃ or position P₄.Operation of motion estimation method 1400 then proceeds, by way ofoff-page connector 1422, to on-page connector 1423.

When execution of motion estimation method 1400 arrives at block 1424,by way of off-page connector 1422 and on-page connector 1423, a steppingdirection 1302-1316 is determined by motion estimation method 1400. Thisstepping direction 1302-1316 is based upon best horizontal positionP_(H), as determined by blocks 1404, 1408, 1410 depending on whether thebest horizontal match is position P_(c), P₁ or P₂. Additionally, thestepping direction 1302-1316 determined in block 1424 is based upon thebest vertical position P_(V) as determined in blocks 1414, 1418, 1420depending on whether the best vertical match was position P_(c), P₃ orP₄.

For example, if the best horizontal position P_(H) selected by motionestimation method 1400 in decision 1402 is position P₁ and the bestvertical position P_(v) is center position P_(c), as determined in block1420, the stepping direction 1302-1316 determined in block 1424 isstepping direction 1314. This results from horizontal motion to the leftand no vertical motion. If, for example, position P₁ is selected indecision 1402 and position P₄ is selected as the best vertical positionP_(V) in decision 1426, stepping direction 1312 is determined in block1424. This results from horizonal motion to the left and downwardvertical motion.

If center position P_(c) is determined to be both the best horizontalposition P_(H) and the best vertical position P_(V) within motionestimation method 1400, it will be understood that the best match fromone frame to the next may be achieved with no motion. This indicates nomotion of positions P_(c), P₁ -P₄ from one frame to the next. Thus thereare nine possible outcomes of motion estimation method 1400 even thoughonly eight stepping directions 1302-1316 are defined. The selection ofstepping direction 1302-1316, in accordance with best vertical positionsP_(V) and best horizontal position V_(H), is set forth in Table VI.

                  TABLE VI                                                        ______________________________________                                        P.sub.V       P.sub.H                                                                             Motion Vector                                             ______________________________________                                        P.sub.c       P.sub.c                                                                             N/A                                                       P.sub.c       P.sub.1                                                                             1314                                                      P.sub.c       P.sub.2                                                                             1306                                                      P.sub.3       P.sub.c                                                                             1302                                                      P.sub.3       P.sub.1                                                                             1316                                                      P.sub.3       P.sub.2                                                                             1304                                                      P.sub.4       P.sub.c                                                                             1310                                                      P.sub.4       P.sub.1                                                                             1312                                                      P.sub.4       P.sub.2                                                                             1308                                                      ______________________________________                                    

When stepping direction 1302-1316 is determined in block 1424 of motionestimation method 1400, the operation of motion estimation method 1400includes movement in the direction of stepping direction 1302-1316formed therein as shown in block 1426. A determination is then made indecision 1428 whether the movement indicated in block 1426 results in animprovement in the least mean square error from one frame to the next.If no improvement is obtained, it may be determined that the least meansquare error has been minimized and that the best match has been found.When the best match has been found, as indicated by decision 1428,execution of motion estimation method 1400 proceeds to end terminal 1432by way of path 1429. At this point motion estimation method 1400 mayhave determined the minimum error between the image to be matched andthe reconstructed image.

If movement in the direction indicated stepping direction 1302-1316 asdetermined in block 1424 results in improvement of the match betweenframes, as indicated for example by determining the least mean squareerror, a determination is made in decision 1430 whether motionestimation method 1400 has timed out. The time out duration of decision1430 may be based upon a predetermined number of processor cycles orupon a predetermined amount of time during which video processing system100. This predetermined number of processor cycles is the numberallotted for video processor 112 to search for the best match betweenthe images of one frame and another. During this predetermined duration,motion estimation method 1400 may iteratively approach the best match.

If the best match occurs prior to the time out execution leaves motionestimation method 1400 as previously described. However, if the time outoccurs while the match is still improving no more iterations of method1400 are permitted and execution of method 1400 proceeds to end terminal1382. Thus motion estimation 1400 continues to search for the best matchonly until the time out occurs. It will be understood that the durationof time out decision 1430 may be dynamically adjusted according to howbusy video processor 112 is. It will also be understood that the timeout test of decision 1430 may be performed along with any type of motionestimation method or any type of method or system for finding a bestmatch. It may be used with various different search strategies which maymaximize or minimize various correlation functions.

If the match between frames continues to improve and there is more timeto search for the best match, as determined in decisions 1428, 1430,execution of motion estimation method 1400 proceeds by way of off-pageconnector 1434 to on page connector 1401. From on-page connector 1401,motion estimation method again searches for the best horizontal matchP_(H), in decisions 1402, 1406, and the best vertical match P_(V), indecisions 1412, 1416, as previously described. When best horizontal andvertical positions P_(H), P_(V) are determined by decisions 1402, 1406,1412, 1416, a new stepping direction 1302-1316 is determined in block1424 and a determination is again made whether movement in the directionindicated by new stepping direction 1302-1316 results in an improvement.

Motion estimation method 1440 is an alternate embodiment of motionestimation method 1400 as previously described. In this alternateembodiment of motion estimation method 1400, execution proceeds fromoff-page connector 1422 to on-page connector 1442 of method 1440. Whenexecution of motion estimation method 1440 proceeds by way of on-pageconnector 1442, a determination is made in decision 1444 whether centerposition P_(c) provides the best match in both the horizontal andvertical directions.

If the determination of decision 1444 is affirmative, this is anindication that no motion vector 1302-1316 provides an improvement. Inthis case no further searching is performed in motion estimation method1400. However, it is possible that motion estimation method 1400 hasfound a local minimum error which is a worse match than some otherundetermined local minima. Therefore, in alternate embodiment motionestimation method 1440, a random point is selected in block 1448. Motionestimation method 1400 is then applied beginning at this random point.

Thus, after the random point is selected in block 1448, executionproceeds from alternate embodiment 1440, by way of off-page connector1449, to on-page connector 1429 of motion estimation method 1400. Atthis point, a determination is made whether movement to the random pointselected in block 1448 provides an improvement in decision 1428.

A pseudocode representation of motion estimation method 1400 is shown inTable VII. In the pseudocode of Table VII execution proceeds from thetime out check to the beginning of the routine to permit continuedstepping in a direction which causes an improved match until no moreimprovement results.

                  TABLE VII                                                       ______________________________________                                        START:                                                                        Try P.sub.1                                                                   If P.sub.1 is worse than P.sub.c                                                       try P.sub.2                                                          Try P.sub.3                                                                   If P.sub.3 is worse than P.sub.c                                                       try P.sub.4                                                          Determine a stepping direction from                                                    best of P.sub.1, P.sub.2, P.sub.c                                             best of P.sub.3, P.sub.4, P.sub.c                                    Step in stepping direction until                                                       no more improvement with time out                                             check.                                                               Go to START                                                                   ______________________________________                                    

Referring now to FIG. 15, there is shown improved dequantization system1500. It will be understood that improved dequantization system 1500 maybe used to provide a portion of the operations performed withindequantization block 222 of compression/decompression accelerator 120within remote video processing system 100 of the present invention.Dequantization system 1500 may perform dequantization upon the datareceived from dequantization multiplexer 218 and apply dequantized datato selectable discrete cosine transform device 230 using only twodecrement operations and one multiplication operation. Thus theoperations of improved dequantization system 1500 are performed morequickly than the dequantization operations of convention dequantizerswhich are believed to require at least two additions and onemultiplication. This improvement in dequantization speed results becausethe decrements of dequantization system 1500 may be performed morequickly than the additions of conventional dequantization systems.

Improved dequantization system 1500 assumes that value≠0 in Equations(2) because zero is not a legal input into system 1500. Dequantizationsystem 1500 also assumes that 1≦Q≦31 and -127≦value≦+127. In order todevelop the computational flow of dequantization system 1500,dequantization Equations (2) may be rewritten as follows: ##EQU7## Itwill be seen from Equations (10) that the two inputs into a multiplierin a system performing dequantization according to Equations (10) are(a) Q, and (b) 2*value+sign(value). It will be understood by thoseskilled in the art that the multiplier input quantity2*value+sign(value), as set forth in Equations (10), must be betweennegative two hundred fifty-five and positive two hundred and fifty-five.Furthermore, it should be noted that the quantity may be expressed asset forth in Equations (11) when value is even. It will also beunderstood that 2*value must be even and that 2*value+1 must be odd.##EQU8## The result of the multiplication Q*(2*value+sign(value)) ofEquations (10) is set forth in Equations (12) ##EQU9##

Thus the eight bit input, value, is received by dequantization system1500 by way of dequantization input bus 1502 and applied to conditionaldecrementing device 1506. The sign bit of the value of input bus 1502 isapplied to sign bit input 1510 of decrementing device 1506. Conditionaldecrementing device 1506 either decrements the value received by way ofinput bus 1502 or passes it through unchanged depending on the sign bitof the input value as received at sign bit input 1510. Conditionaldecrementing device 1506 applies the result, either decremented orpassed straight through, to output bus 1514.

At point 1518 of the computation flow of dequantization device 1500, theeight bit output of conditional decrementing device 1506 is shifted tothe right one bit position. A value of one is inserted into the leastsignificant bit of the nine bit value resulting therefrom. It will beunderstood that this operation provides the results of the calculationsset forth in dequantization Equations (11). In the preferred embodimentof dequantization device 1500 no specialized shifting circuitry isrequired to perform the operations associated with point 1518. Rather,in the preferred embodiment of dequantization device 1500, this resultis obtained be merely wiring the eight bits from the output ofdecrementing device 1514 to the next higher order bits at input 1524 ofmultiplication device 1526. The least significant bit of this inputvalue is hardwired high.

Multiplication device 1526 receives quantization factor Q by way ofmultiplication input 1522, as well as receiving the shifted output ofdevice of conditional decrementing device 1506 by way of multiplicationinput 1524. These two inputs are multiplied within multiplication device1524 to provide the fourteen bit output quantity represented as multoutin dequantization Equations (12). This quantity appears atmultiplication output 1530.

At point 1534 of dequantization system 1500 the quantity multout,provided by multiplication device 1526, is shifted one position to theleft to provide the quantity multout/2 as set forth in Equations (12).This value is then applied, by way of input bus 1538, to conditionaldecrementing device 1546. Conditional decrementing device 1546 receivesthe inverse of the sign bit of value, as received by input bus 1502, atsign bit input 1542. Conditional decrementing device 1546 eitherdecrements the quantity received on input bus 1538 or passes it throughunchanged according to the inverted sign bit of value. The thirteen bitresult of this conditional decrement operation is provided bydecrementing device 1546 at output bus 1550. A hardwired value of one isapplied to the signal of output bus 1550 at point 1554. The fourteen bitresult is applied to block 1558 where it may be clipped, and wherespecial cases such as zero value inputs and interblock DC encoding maybe handled.

Referring now to FIG. 16, there is shown a more detailed representationof the various memories and controls associated with bus interface 200.Bus interface 200 includes four buffer memories 204, 206, 240, 248.Memories 204, 206, 240, 248 buffer the incoming data to be processed bycompression/decompression system 120 as well as the results ofoperations performed by compression/decompression system 120 which arewritten to other devices within remote video processing system 100 usingbuses 116, 118.

Loop filter memory 206 stores the previous image block for both theencode and decode processes of accelerator 120 even when selectable loopfilter 210 is disabled. Memory 206 is also used to store intermediateresults of operations performed by loop filter 210. Thus it is ten bitswide in the preferred embodiment in order to accommodate two fractionbits required by loop filter 210 in addition to the eight bits of pixeldata. When an image is initially loaded into loop filter memory 206, theeight bits of each image pixel are placed in the most significant eightbits of a location within loop filter memory 206. The least significanttwo bits set to zero.

Loop filter memory 206 is two-ported, with one read port 1604 and onewrite port 1602. Data is applied to write port 1602 by way of data line203 and data is read from port 1604 by way of data line 207 aspreviously described. Write port 1602 of loop filter memory 206 has twosources of data. One source of data is bus interface 200 itself when anew image is loaded into compression/decompression accelerator 120. Theother source of data is selectable loop filter 210 when intermediatefilter results are generated. Loop filter 210 provides addresses oflocations within loop filter memory 206 for its writes. Loop filtermemory 206 increments write addresses itself during the load of imagedata from bus interface 200. Reads of loop filter memory 206 areasynchronous.

Run length decoder/current image memory 204 serves as an input bufferfor two different functions. During the decode mode ofcompression/decompression accelerator 120, image memory 204 stores theincoming run/value pairs which are decoded by run length decoder 208.During the encode process the current image block to be encoded isstored in image memory 204.

In the preferred embodiment of accelerator 120 image memory 204 may havea single physical read/write port. The read port uses one of twoaddresses supplied to image memory 204. One address used by image memory204 is provided by run length decoder 208 during decode in order toselect data for decoding. The other address is provided by bus interface200 during the encode mode of accelerator 120. Output data of imagememory 204 is provided by way of line 205 in both a single byte formatand in a sixteen bit format.

During the decode mode of accelerator 120, up to sixty-four run/valuepairs may be decoded by run length decoder 208. There is a capacity forthirty-one sixteen bit run/value pairs plus the end of block code. Ifthirty-two or more run/value pairs occur in a block, run length decoder208 pauses while the additional data is loaded. This is accomplishedwith a halt signal, asserted by image memory 204 when run length decoder208 requests the thirty-third word, thereby indicating that the addresshas wrapped from thirty to zero.

Decoded image memory 240 stores the decoded image block which is writtento memory external to compression/decompression accelerator 120, such asmemory 114. When accelerator 120 is in the encode mode, this datarepresents the companded image which is stored in current/compandedimage block 302. Decoded image memory 240 is provided with one portwhich is loaded one byte at a time by frame adder 235. The single portof memory 240 is read four bytes at a time by bus interface 200 whenaccelerator 120 writes companded data onto data bus of remote videoprocessing system 100 in order to permit motion estimation to beperformed by video processor 112. The internal address incrementer ofimage memory 240 may be externally reset.

Zigzag memory 248 stores the run/value pairs generated cooperatively byzigzag/quantization block 238 and run length encode block 246 aspreviously described. The run/value pairs are read from zigzag memory248 by bus interface 200 for writing data bus of remote video processingsystem 100. Memory 248 contains its own address incrementer.

Bus interface 200 is also provided with global control block 260 whichactivates the other elements associated with bus interface 200 such asmemories 204, 206, 240, 248. Most activities of elements activated byglobal control block 210 are completed in a predetermined number ofclock cycles. For these elements, global control block 260 counts clockcycles to determine the end of a control state. The activities of theexternal bus interface are tracked by an end-of-process signal receivedfrom bus interface 200 by way of master control block 262. In additionto this end-of-process signal and the clock inputs received by globalcontrol block 260 from master control 262 include the system reset andhalt control signals of compression/decompression accelerator 120. Whena halt control signal is received the clock of global control block 260is suspended.

Referring now to FIG. 17, there is shown a block diagram representationof selectable frame add/subtract unit 1700 of compression/decompressionaccelerator 120. It will be understood that selectable frameadd/subtract unit 1700 is effective to selectably provide the operationsof both frame difference block 220 and frame add block 235 ofaccelerator 120. These operations are selected under the control ofcontrol lines 226, 228 depending on whether accelerator 120 is operatingin the encode mode or the decode mode and, when accelerator 120 isoperating in the encode mode, whether encoded data is being prepared tobe transmitted or to be decoded to provide a companded image.

As previously described, frame difference block 220 and frame add block235 perform several functions within compression/decompressionaccelerator 120. Frame difference block 220 provides selectable discretecosine transform device 230 with motion estimation data by way ofmultiplexer 224 for encoding. Frame add block 235 generates compandedimage data during the encode mode of accelerator 120 and applies thecompanded image data to decoded image memory 240. During the decode modeof accelerator 120 blocks 220, 235 reconstruct the current image incurrent/companded image block 302. Furthermore, blocks 220, 235 preventquantization distortion of transform coefficient amplitude that maycause arithmetic overflow by providing saturation functions.

In the preferred embodiment of compression/decompression accelerator 120frame difference block 220 and frame add block 235 are combined into asingle hardware circuit. This is possible because the frame add andframe subtract functions of accelerator 120 are never performedsimultaneously. Frame add/subtract unit 1700 is used during both theencode and decode modes of accelerator 120. In the encode mode, it isnecessary to reconstruct the image as seen by a receiving device inorder for video processor 112 to determine how the next block of datashould be encoded. For an inter block, the companded image is generatedby adding previous block memory to the encoded difference as received bythe receiving device after discrete cosine transforms, quantization anddequantization. The addition is done by the frame add hardware.

In the decode mode of accelerator 120, the reconstruction of an imagefor an inter block is done by adding the previous predicted pixel blockwith the decoded image data or difference data. Frame addition and framesubtraction are not required for intra block data since there are nodependencies involved with previous image.

To prevent quantization distortion of transform coefficient amplitudescausing arithmetic overflow within encode dataflow 300 and decodedataflow 600, a clipping or saturate function is included inadd/subtract unit 1700. The saturation function is applied to areconstructed image which is formed by summing the prediction and theprediction error during the encoding mode of compression/decompressionaccelerator 120. Results are saturated between zero and two hundredfifty-five.

In order to optimize resources and increase throughput, the frameaddition in the decode mode and frame subtraction in the encode mode aredone simultaneously with the horizontal filter cycle or second filtercycle of loop filter 210. Since the horizontal cycle of loop filter 210scans the eight-by-eight image block according to row, the current blockdata to be added or subtracted from it must also be fetched according torow. Global control block 260 ensures that the horizontal pass does notstart until the current image data is ready.

Referring now to FIG. 18, there is shown loop filter state diagram 1800representing the state of selectable loop filter 210 as well as a moredetailed block diagram representation of portions of selectable loopfilter 210. Selectable loop filter state diagram 1800 represents thestates of selectable loop filter 210 when compression/decompressionaccelerator operates in the encode mode as described in encode dataflow300.

Selectable loop filter 210 is a two-dimensional low-pass spatial filterwhich operates on predicted eight-by-eight blocks of a video image aspreviously described. Filter 210 smooths video data and reduces highfrequency content in order to produce a better compression ratio duringthe discrete cosine transform of selectable transform device 230 and therun length encoding of encoder block 246. The improvement in thecompression ratio is at the expense of resolution or image sharpnessbecause of the removal of higher frequencies. Selectable loop filter 210may be enabled and disabled when encoding or decoding an image usingcontrol line 214.

In the filtering process of selectable loop filter 210, the value of apixel is adjusted by averaging it with the values of adjacent pixels.Different weights are assigned to the adjacent pixels as previouslyshown in Table I. The pixel being filtered is centered and highlightedin Table I and is given the highest weight of the nine pixels involvedin the averaging.

Simultaneously applying a loop filter such as selectable loop filter 210in two dimensions is difficult. However, it can be shown that thefiltering process is symmetrical and can be separated into twoone-dimensional processes that are much easier to implement. Thus thetwo-dimensional filtering process is separated into a one-dimensionalhorizontal function and a one-dimensional vertical non-recursivefunction in the preferred embodiment. The values 1/4, 1/2 and 1/4 areused as the coefficients for each one-dimensional function. However, atthe block edges where one or more points or fall outside the block onlyone-dimensional filtering is applied. Filtering is not applied to thefour corner pixels.

When performing the filtering of selectable loop filter 210, theeight-by-eight predicted pixel block is organized in a two-dimensionalarray of eight rows and eight columns of eight bit data. This data isstored in loop filter memory 206 as an eight-by-eight ten bit block. Theten bits per pixel in loop filter memory 206 are necessary to maintainthe required resolution during the immediate filtering step aspreviously described. Loop filter 210 first operates upon the eightcolumns, performing a one-dimensional vertical filtering on each. Theintermediate results generated by this vertical filtering process arestored back into loop filter memory 206 from which the original data isread. The read and write addresses are generated by address generator264.

Upon the completion of the vertical filter cycle, selectable loop filter210 scans the two-dimensional array being filtered row by row to performthe horizontal filter process. The result of this process is a stream oftwelve bit precision values which are rounded to eight bits. This resultis applied directly to frame add/subtract unit 1700 in addition to beingwritten back to the loop filter memory 206. Simultaneously, with thesecond or horizontal pass output, data is read from current image memory204. The read address for memory 204 is the same as the write addressprovided by loop filter 210. The two streams of eight bit values aresent to frame add/subtract unit 1700.

Selectable loop filter 210 may begin this filtering process when loopfilter memory 206 is filled. Before this time, loop filter 210 idles inidle state 1828 of loop filter state diagram 1800. Selectable loopfilter 210 enters reset/idle state 1828 whenever it is initialized inorder to prepare for a new loop filter cycle. While in state 1828 theaddress counter is set to zero and the horizontal/vertical filter stateis set to vertical.

When the filtering process is begun, vertical filter state 1844 isentered by way of pathway 1832. The first pass, or the vertical pass, ofloop filter 210 scans an eight-by-eight input matrix of pixel valuesstarting from the top of the matrix and proceeding to the bottom. Thesevertical scans start at the top left corner of the matrix which may bedesignated location 0, 0. Three buffer registers 1902, 1904, 1906 areused to store the pixels at the input of filter adder 1908. When firstregister 1902 and second register 1904 are filled, processing by loopfilter 210 begins and the partially processed values are written backloop filter memory 206 by way of data line 209. At the end of thevertical state 1844, loop filter memory 206 is filled with sixty-fourten bit intermediate values. The bottom two bits of each intermediatevalue are the fractional bits which maintain precision for the next passof loop filter 210.

After the vertical pass of loop filter memory 206 is complete,selectable loop filter 210 pauses and waits for control signals fromglobal control block 260. If a reset occurs during this time, loopfilter 210 enters idle/reset state 1828 by way of reset path 1840. If noreset occurs during this time, the control signal beginning of thehorizontal filter cycle is provided by global control block 260 when (1)frame memory 204 is full, (2) selectable transform device 230 is readyto receive data from frame add/subtract unit 1700, and (3) the verticalpass cycle is complete.

When the control signal from control block 260 is received by selectableloop filter 210 horizontal filter state 1852 of loop filter statediagram 1800 is entered by way of path 1848. During the horizontal passof selectable loop filter 210 the matrix of intermediate loop results inloop filter memory 206 is read in the left to right horizontaldirection. Data is written back to loop filter memory 206 in the mannerpreviously described with respect to the results of the vertical pass.Data is also sent to frame difference block 220 or selectable frameadd/subtract unit 1700 simultaneously when compression/decompressionaccelerator 120 is operating in the encode mode.

Loop filter 210 returns to idle state 1828 of state diagram 1800 by wayof path 1856 when the horizontal filtering is complete or a resetoccurs. Selectable loop filter 210 remains in reset/idle state 1828until either (1) a vertical control signal is asserted instructing loopfilter 210 to exit state 1828 by way of pathway 1832, or (2) logicwithin global control 260 decodes a filter bypass command therebyindicating an exit from state 1828 by way of pathway 1820. All othersignals are ignored by loop filter 210 in reset/idle state 1828.

In filter bypass state 1808 of filter state diagram 1800, no filteringis performed by selectable loop filter 210. When loop filter 210 is inbypass state 1808 adder 1908 is disabled. Address generator 264 producesaddresses to permit frame add/subtract unit 1700 to write to selectablediscrete cosine transform device 230 or decoded image memory 240. Datais read from loop filter memory 206 and current image memory 204 andapplied to selectable transform block 230. Bypass state 1808 is enteredby way of path 1820 from idle state 1828. Selectable loop filter 210exits bypass state 1808 when a reset occurs or it has reached the end ofa block.

Address generator 2002 of loop filter 210 generates all the addressingfor loop filter memory 206 and current image memory 204 accesses. Theaddresses are generated sequentially and range from 00 H to 3 fH.Address generator has three main functions. The first function ofaddress generator 2000 is to generate horizontal pass addresses for loopfilter 210. Generator 2002 produces sequentially ascending addressesfrom 00 H to 3 fH during the horizontal pass. Whenever a corner or sideaddress is processed by adder 1900 the output of adder 900 is ignoredand the contents of 1904 are copied instead. The addresses generated areused to read from loop filter memory 206 and to read from current imagememory 204.

Another function of address generator 2002 is generating vertical passaddresses for loop filter 210. In this mode, the upper three bits areswapped with the lower three bits of the address generated to produce atop-to-bottom scan of the eight-by-eight matrix by loop filter 210. Theswapping is performed using multiplexer 2004 whenever a corner, a toprow or a bottom row address of the pixel matrix is processed by adder1908. When this occurs the output of adder 1908 is ignored and thecontents of REG2 are copied over instead. The addresses are used to readfrom loop filter memory 206 and to write back to loop filter memory 206.Current image memory 204 is not accessed during this pass.

Another function of address generator 2002 is generating read addressesof loop filter memory 206 when selectable loop filter 210 is bypassedwithin accelerator 120. When this occurs adder 1900 is disabled and theaddresses generated are used to read of loop filter memory 206 andcurrent image memory 204. The data is sent to the frame add/subtractblock 1700 after being sent through disabled adder 1908. The addressesgenerated are all in the horizontal read mode, from left to right.

A determination is made when a corner, a top row, or a bottom row isbeing processed dividing the output of counter 2002 into two sets ofthree bits each using buses 2003. Buses 2003 are applied to multiplexer2004 and wrapped. Thus, in summary, address generator 2000 of selectablefilter 310 includes six bit counter 2002, two registers 2006, 2008 andbyte-swap multiplexer 2004. Address register 200 generates the addressesin loop filter memory 206 from which the filter input data is read fromand the addresses where the intermediate filtered result are stored. Italso generates addresses for reading from current image memory 204, forwriting to selectable discrete cosine transform device 230, and forwriting decoded image memory 240. During the vertical pass of loopfilter 210, the upper three bits of the address are swapped with thelower three bits by multiplexer 2004. This results in a top-to-bottomread instead of a left-to-right read of the eight-by-eight pixel matrixbeing processed by loop filter 210.

Adder unit 2000 includes three input twelve bit adder 1908. In additionto three twelve bit inputs adder 1908 has a single output. Two of theinputs of adder 1900 are connected directly to the outputs of registers1902, 1906 the third input of adder 1908 is from register 1904. Register1904 contains the pixel that is currently being processed. Registers1902, 1904, 1906 are standard ten bit registers. For the first pass, thecontains only eight bit values and two extra bits are padded on. Theoutput of REG1 and REG3 goes directly to the adder. REG2 goes through amultiply-by-two operation before going to adder 1908. The multiplyoperation is done by mapping the bits up by one.

If the pixel being processed is a corner/side pixel that requires nofiltering, the output from adder 1908 is ignored and the contents ofregister 1904 are written directly to loop filter memory 206. Thedecision whether to writing or to bypass is made by logic located at theoutput of address generator 2000. This also occurs when selectable loopfilter 210 is in filter bypass state 1802 of state diagram 1800.

Thus, in summary, adder unit 1900 of selectable loop filter 210 includesthree registers 1902, 1904, 1902 and two two-input twelve bit adders inseries. Three consecutive inputs to filter 210 are queued in a pipelineformed by register 1902, 1904, 1906. Adder circuit 1908 performs theone-dimensional filter process by performing a simple add register 1902,1906, and a double add for register 1904. During the first or verticalpass of loop filter 210 three eight-bit values are read first or and aten-bit sum is written back to loop filter memory 206. During the secondor horizontal pass of loop filter 210 the ten-bit values are read backfrom loop filter memory 206 and added to result in a twelve bit value.This result is rounded up to eight bits before sending it to the frameadd/subtract unit 1700. Values whose fractional part are one-half arerounded up.

Controller 213 is responsible for handling the various inputs from otherunits and decoding them to provide controls for adder unit 1900 andaddress generator 200. For example, controller 213 transmits controlsignals to address generator 2000 to start counting. It also enables anddisables adder 1908.

Based on control signals received by controller 213, controller 213 mayput loop filter 210 into the bypass mode. Filter 210 then acts as anaddress generator for frame add/subtract unit 1700.

Loop filter controller 213 of selectable loop filter 210 includes astate machine not shown and some miscellaneous logic. The state machineof controller 213 sequences the various filter and read/write processes.

It will be understood that various changes in the embodiments which havebeen described and illustrated in order to explain the nature of thisinvention may be made by those skilled in the art without departing fromthe principle and scope of the invention as expressed in the followingclaims.

I claim:
 1. A method for estimating image motion in a video processingsystem having a test image, a plurality of candidate images, differencedetermining means for determining a difference between images, andduration determining means for determining the duration of the imagemotion estimation process of steps (a), (b), and (c) below, comprisingthe steps of:(a) selecting a candidate image of said plurality ofcandidate images; (b) determining a difference between said test imageand said selected candidate image; (c) estimating said image motion inaccordance with the determining of step (b); (d) determining theduration of the image motion estimation process of steps (a), (b), and(c) above, by said duration determining means; and, (e) repeating step(c) in accordance with the duration determination of step (d).
 2. To themotion estimation method of claim 1, wherein said determined differencedecreases as step (c) is repeated.
 3. The motion estimation method ofclaim 2, comprising the further steps of:(f) determining whether saiddifference continues to decrease when step (c) is repeated; and, (g)repeating step (c) in accordance with the determination of step (f). 4.The motion estimation method of claim 3, comprising the step of randomlyselecting a candidate image from said plurality of candidate images inaccordance with the determination of step (f).
 5. The motion estimationmethod of claim 1, further comprising the step of comparing saiddetermined duration with a predetermined duration.
 6. The motionestimation method of claim 5, further comprising the step of comparingsaid duration with a predetermined time duration.
 7. The motionestimation method of claim 5, wherein said video processor system hasvideo processor system cycles further comprising the step of comparingsaid duration with a predetermined number of video processor systemcycles.
 8. The motion estimation method of claim 5, wherein saidpredetermined time duration is varied.
 9. The motion estimation methodof claim 8, wherein said predetermined duration is varied in accordancewith utilization of said video processor system.
 10. The motionestimation method of claim 1, further comprising the step of determininga motion vector in accordance with the determined difference of step(b).
 11. The motion estimation method of claim 10, wherein saidcandidate image of step (a) is selected in accordance with said motionvector.
 12. The motion estimation method of claim 10, wherein step (b)comprises the step of determining a plurality of differences and saidmotion vector is determined in accordance with said plurality ofdifferences.
 13. The motion estimation method of claim 1, wherein step(a) comprises selecting candidate images having first and secondrelative displacements from a center image having a predeterminedstarting image position.
 14. The motion estimation method of claim 13,comprising the further steps of:(f) determining relative errors of saidcandidate images having said first displacement from said center image;and, (g) determining relative errors of said candidate images havingsaid second displacement from said center image.
 15. The motionestimation method of claim 14, comprising the further step of selectingsaid candidate image in accordance with the determinations of steps (f)and (g).
 16. The motion estimation method of claim 13, wherein saidfirst and second relative displacements are perpendicular to each other.